IN THE CLAIMS 

The following is a complete listing of the claims, and replaces all earlier 
versions and listings. 

1 . (Previously Presented) A method of processing image data defining 
a plurality of sequences of images, each from a respective camera, of a plurality of objects 
moving in a scene to produce signals defining representations of the objects in a three- 
dimensional computer model, the method comprising the steps of: 

processing image data from a first of the cameras to identify image 
data relating to objects in the scene; 

processing image data from a second of the cameras to identify 
image data relating to objects in the scene; 

processing the identified image data from the first camera for each 
object to define an object representation in the three-dimensional computer model having a 
height dependent upon the image data for the object from the first camera; 

processing the identified image data from the second camera for 
each object to define an object representation in the three-dimensional computer model 
having a height dependent upon the image data for the object from the second camera; 

comparing the height of the representation of each object generated 
in dependence upon image data from the first camera with the height of the representation 
of the corresponding object generated in dependence upon image data from the second 
camera; and 



generating object representations in the three-dimensional computer 
model in dependence upon the height comparisons, 

wherein, when the heights of the corresponding representations are 
not within a predetermined amount of each other, the taller representation is split into a 
first portion having a height corresponding to the height of the smaller representation and a 
second portion comprising the remaining part of the taller representation, and wherein a 
further representation is defined in the three-dimensional model by re-positioning the 
second portion in the three-dimensional model. 

2.-6. (Cancelled) 

7. (Previously Presented) A method according to claim 1, wherein the 
second portion is re-positioned in dependence upon a representation defined on the basis of 
image data from the camera which produced the smaller representation. 

8. (Original) A method according to claim 7, wherein the second 
portion is re-positioned by: 

identifying which of the representations defined on the basis of 
image data from the camera which produced the smaller representation overlaps the image 
data used to define the taller representation in the image space of the camera which 
produced the taller representation; and 

re-positioning the second portion, in dependence upon the position 
of the identified representation in the three-dimensional model. 
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9. (Original) A method according to claim 8, wherein the second 
portion is re-positioned by: 

mapping at least part of each representation defined on the basis of 
image data from the camera which produced the smaller representation from the 
three-dimensional model to the image space of the camera which produced the taller 
representation; 

determining which projected representation overlaps the image data 
for the taller representation in the image space of the camera which produced the taller 
representation; and 

re-positioning the second portion in dependence upon the position in 
the three-dimensional model of the representation which, when projected into the image 
space of the camera which produced the taller representation, overlapped the image data for 
the taller representation. 

10. (Original) A method according to claim 9, wherein the second 
portion is re-positioned so that the centre of its base is at the same position as the centre of 
the base of the representation which overlapped the image data for the taller representation. 

1 1 . (Previously Presented) A method according to claim 1 , wherein 
each object representation is defined as a planar surface with its base on a predetermined 
surface in the three-dimensional computer model and with a position and size in 
dependence upon a polygon bounding the image data for the object. 
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12. (Original) A method according to claim 11, wherein the polygon is 

a rectangle. 

13. (Original) A method according to claim 12, wherein the sides of the 
rectangle are parallel to the sides of the image. 

14. (Original) A method according to claim 11, wherein the width of 
the planar surface is determined by the width of the bounding polygon in the image data, 
and the height of the planar surface is calculated using the aspect ratio of the bounding 
polygon in the image data. 

15. (Original) A method according to claim 11, wherein the planar 
surface lies within a vertical plane. 

16. (Original) A method according to claim 1, further comprising the 
step of generating image data by rendering an image of the three-dimensional computer 
model in which texture data based on the processed image data is rendered onto the 
representation of each object. 

17. (Original) A method according to claim 16, further comprising the 
step of generating a signal conveying the image data. 
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18. (Original) A method according to claim 17, further comprising the 
step of recording the signal. 

19. (Original) A method according to claim 16, further comprising the 
step of displaying an image of the objects using the generated image data. 

20. (Original) A method according to claim 16, further comprising the 
step of making a recording of the image data either directly or indirectly. 

2 1 . (Previously Presented) A method of image processing in which 
image data from first and second cameras is processed to identify image data relating to 
respective objects, the height of each object in a modelling space is determined using the 
identified image data, and the heights of objects determined using image data from the first 
camera are compared with the heights of objects determined using image data from the 
second camera to determine which if any identified image data relates to more than one 
object, 

wherein each object is defined as a planar surface with its base on a 
predetermined surface in the modelling space and with a position in dependence upon a 
polygon bounding the image data for the object, and wherein the width of the planar 
surface is determined by the width of the bounding polygon in the image data, and the 
height of the planar surface is calculated using the aspect ratio of the bounding polygon in 
the image data. 
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22. (Original) An image processing method in which image data from a 
first camera of objects in a scene is processed to identify image data relating to respective 
objects, and image data from a second camera of the objects in the scene is processed to 
determine whether any of the identified image data from the first camera relates to more 
than one object by comparing a size parameter of each object determined from the image 
data of the first camera with the corresponding size parameter determined from the image 
data of the second camera. 

23. (Previously Presented) An image processing apparatus for 
processing image data defining a plurality of sequences of images, each from a respective 
camera, of a plurality of objects moving in a scene to produce signals defining 
representations of the objects in a three-dimensional computer model, the apparatus 
comprising: 

means for processing image data from a first of the cameras to 
identify image data relating to objects in the scene; 

means for processing image data from a second of the cameras to 
identify image data relating to objects in the scene; 

means for processing the identified image data from the first camera 
for each object to define an object representation in the three-dimensional computer model 
having a height dependent upon the image data for the object from the first camera; 

means for processing the identified image data from the second 
camera for each object to define an object representation in the three-dimensional computer 
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model having a height dependent upon the image data for the object from the second 
camera; 

means for comparing the height of the representation of each object 
generated in dependence upon image data from the first camera with the height of the 
representation of the corresponding object generated in dependence upon image data from 
the second camera; and 

means for generating object representations in the three-dimensional 
computer model in dependence upon the height comparisons, 

wherein the apparatus is arranged to perform processing such that, 
when the heights of the corresponding representations are not within a predetermined 
amount of each other, the taller representation is split into a first portion having a height 
corresponding to the height of the smaller representation and a second portion comprising 
the remaining part of the taller representation, and wherein a further representation is 
defined in the three-dimensional model by re-positioning the second portion in the three- 
dimensional model. 

24. - 28. (Cancelled) 

29. (Previously Presented) Apparatus according to claim 23, arranged to 
perform processing such that the second portion is re-positioned in dependence upon a 
representation defined on the basis of image data from the camera which produced the 
smaller representation. 
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30. (Original) Apparatus according to claim 29, arranged to perform 
processing such that the second portion is re-positioned by: 

identifying which of the representations defined on the basis of 
image data from the camera which produced the smaller representation overlaps the image 
data used to define the taller representation in the image space of the camera which 
produced the taller representation; and 

re-positioning the second portion in dependence upon the position of 
the identified representation in the three-dimensional model. 

31. (Original) Apparatus according to claim 30, arranged to perform 
processing such that the second portion is re-positioned by: 

mapping at least part of each representation defined on the basis of 
image data from the camera which produced the smaller representation from the 
three-dimensional model to the image space of the camera which produced the taller 
representation; 

determining which projected representation overlaps the image data 
for the taller representation in the image space of the camera which produced the taller 
representation; and 

re-positioning the second portion in dependence upon the position in 
the three-dimensional model of the representation which, when projected into the image 
space of the camera which produced the taller representation, overlapped the image data for 
the taller representation. 
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32. (Original) Apparatus according to claim 31, arranged to perform 
processing such that the second portion is re-positioned so that the centre of its base is at 
the same position as the centre of the base of the representation which overlapped the 
image data for the taller representation. 

33. (Previously Presented) Apparatus according to claim 23, arranged to 
perform processing such that each object representation is defined as a planar surface with 
its base on a predetermined surface in the three-dimensional computer model and with a 
position and size in dependence upon a polygon bounding the image data for the object. 

34. (Original) Apparatus according to claim 33, wherein the polygon is 

a rectangle. 

35. (Original) Apparatus according to claim 34, wherein the sides of the 
rectangle are parallel to the sides of the image. 

36. (Original) Apparatus according to claim 33, arranged to perform 
processing such that the width of the planar surface is determined by the width of the 
bounding polygon in the image data, and the height of the planar surface is calculated using 
the aspect ratio of the bounding polygon in the image data. 

37. (Original) Apparatus according to claim 33, arranged to perform 
processing such that the planar surface lies within a vertical plane. 
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38. (Original) Apparatus according to claim 23, further comprising 
means for generating image data by rendering an image of the three-dimensional computer 
model in which texture data based on the processed image data is rendered onto the 
representation of each object. 

39. (Original) Apparatus according to claim 38 further comprising 
means for displaying an image of the objects using the generated image data. 

40. (Previously Presented) An image processing apparatus operable to 
process image data from first and second cameras to identify image data relating to 
respective objects, to determine the height of each object in a modelling space using the 
identified image data, and to compare the heights of objects determined using image data 
from the first camera with the heights of objects determined using image data from the 
second camera to determine which if any identified image data relates to more than one 
object, wherein the apparatus is arranged to perform processing such that each object is 
defined as a planar surface with its base on a predetermined surface in the modelling space 
and with a position in dependence upon a polygon bounding the image data for the object, 
and such that the width of the planar surface is determined by the width of the bounding 
polygon in the image data, and the height of the planar surface is calculated using the 
aspect ratio of the bounding polygon in the image data. 

41 . (Original) An image processing apparatus operable to process image 
data from a first camera of objects in a scene to identify image data relating to respective 
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objects, and to process image data from a second camera of the objects in the scene to 
determine whether any of the identified image data from the first camera relates to more 
than one object by comparing a size parameter of each object determined from the image 
data of the first camera with the corresponding size parameter determined from the image 
data of the second camera. 

42. (Currently Amended) A storage medium storing instructions for 
causing a programmable processing apparatus to become operable to perform a method 
according to any one of claims 1,21 and 22. 

43. (Currently Amended) A signal conveying physically-embodied 
computer program product including instructions in computer-readable form, including 
instructions for causing a programmable processing apparatus to become operable to 
perform a method according to any one of claims 1,21 and 22. 

44. (Original) A method of processing image data defining a plurality of 
sequences of images, each from a respective camera, of an object moving in a scene to 
produce signals defining a representation of the object in a three-dimensional computer 
model, the method comprising: 

processing image data from a first of the cameras to identify image 
data relating to the object in the scene; 

processing image data from a second of the cameras to identify 
image data relating to the object in the scene; 

- 12- 



applying a transformation to the identified image data from the first 
camera which defines a mapping from the ground plane in the space of the image data of 
the first camera to a surface in a modelling space; 

applying a transformation to the identified image data from the 
second camera which defines a mapping from the ground plane in the space of the image 
data of the second camera to the surface in the modelling space; 

comparing the transformed image data from the first and second 
cameras on the surface in the modelling space; 

determining which part of the image data represents shadow in 
dependence upon the comparison results; and 

generating a representation of at least the object in the 
three-dimensional model. 

45. (Original) A method according to claim 44, further comprising the 
step of generating a representation of the shadow in the three-dimensional model. 

46. (Original) A method according to claim 44, wherein the surface in 
the modelling space is the ground plane in the three-dimensional model. 

47. (Original) A method according to claim 44, wherein it is determined 
that aligned parts of the transformed image data represent shadow. 
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48. (Original) A method according to claim 44, further comprising the 
step of generating image data by rendering an image of the three-dimensional computer 
model in which texture data based on the processed image data is rendered onto the 
representation of the object. 

49. (Original) A method according to claim 48, wherein the image data 
rendered onto the representation is determined in dependence upon the comparison results. 

50. (Original) A method according to claim 48, further comprising the 
step of generating a signal conveying the image data. 

51. (Original) A method according to claim 50, further comprising the 
step of recording the signal. 

52. (Original) A method according to claim 48, further comprising the 
step of displaying an image of the object using the generated image data. 

53. (Original) A method according to claim 48, further comprising the 
step of making a recording of the image data either directly or indirectly. 

54. (Original) A method of generating a model of an object in a 
three-dimensional computer model by processing images of the object from a plurality of 
cameras, in which image data from a first camera is processed to identify image data 
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relating to the object and its shadow together, and image data from a second camera is used 
to determine the identified image data from the first camera which relates to the shadow 
and the identified image data from the first camera which relates to the object. 

55. (Previously Presented) A method of generating a model of an object 
in a three-dimensional computer model, in which: 

a transformation is applied to image data from a first camera relating 
to the object and its shadow which maps the image data to a surface to give first 
transformed image data; 

a transformation is applied to image data from a second camera 
relating to the object and its shadow which maps the image data to the surface to give 
second transformed image data; 

the first transformed image data and the second transformed image 
data are compared to distinguish transformed image data relating to the object from 
transformed image data relating to its shadow; and 

the object is modelled in dependence upon the transformed image 
data relating to the object. 

56. (Original) Apparatus for processing image data defining a plurality 
of sequences of images, each from a respective camera, of an object moving in a scene to 
produce signals defining a representation of the object in a three-dimensional computer 
model, the apparatus comprising: 
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means for processing image data from a first of the cameras to 
identify image data relating to the object in the scene; 

means for processing image data from a second of the cameras to 
identify image data relating to the object in the scene; 

means for applying a transformation to the identified image data 
from the first camera which defines a mapping from the ground plane in the space of the 
image data of the first camera to a surface in a modelling space; 

means for applying a transformation to the identified image data 
from the second camera which defines a mapping from the ground plane in the space of the 
image data of the second camera to the surface in the modelling space; 

means for comparing the transformed image data from the first and 
second cameras on the surface in the modelling space; 

means for determining which part of the image data represents 
shadow in dependence upon the comparison results; and 

means for generating a representation of at least the object in the 
three-dimensional model. 

57. (Original) Apparatus according to claim 56, further comprising 
means for generating a representation of the shadow in the three-dimensional model. 

58. (Original) Apparatus according to claim 56, wherein the surface in 
the modelling space is the ground plane in the three-dimensional model. 
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59. (Original) Apparatus according to claim 56, arranged such that it is 
determined that aligned parts of the transformed image data represent shadow. 

60. (Original) Apparatus according to claim 56, further comprising 
means for generating image data by rendering an image of the three-dimensional computer 
model in which texture data based on the processed image data is rendered onto the 
representation of the object. 

61. (Original) Apparatus according to claim 60, wherein the image data 
rendered onto the representation is determined in dependence upon the comparison results. 

62. (Original) Apparatus according to claim 60, further comprising 
means for displaying an image of the object using the generated image data 

63. (Original) Apparatus for generating a model of an object in a 
three-dimensional computer model by processing images of the object from a plurality of 
cameras, the apparatus being operable to process image data from a first camera to identify 
image data relating to the object and its shadow together, and operable to use image data 
from a second camera to determine the identified image data from the first camera which 
relates to the shadow and the identified image data from the first camera which relates to 
the object. 
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64. (Previously Presented) Apparatus for generating a model of an 
object in a three-dimensional computer model, comprising: 

means for applying a transformation to image data from a first 
camera relating to the object and its shadow which maps the image data to a surface to give 
first transformed image data; 

means for applying a transformation to image data from a second 
camera relating to the object and its shadow which maps the image data to the surface to 
give second transformed image data; 

means for comparing the first transformed image data and the 
second transformed image data to distinguish transformed image data relating to the object 
from transformed image data relating to its shadow; and 

means for modelling the object in dependence upon the transformed 
image data relating to the object. 



65. (Currently Amended) A storage medium storing instructions for 
causing a programmable processing apparatus to become operable to perform a method 
according to any one of claims 44, 54 and 55. 



66. (Currently Amended) A signal conveying physically-embodied 
computer program product including instructions in computer-readable form, including 
instructions for causing a programmable processing apparatus to become operable to 
perform a method according to any one of claims 44, 54 and 55. 
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67. (Original) A method of processing image data defining a plurality of 
sequences of images, each from a respective camera, of an object moving in a scene to 
produce signals defining a representation of the object in a three-dimensional computer 
model, the method comprising: 

processing image data from a first of the cameras to identify image 
data relating to the object in the scene; 

processing image data from a second of the cameras to identify 
image data relating to the object in the scene; 

processing the identified image data from the first camera and the 
identified image data from the second camera to determine a footprint of the object on the 
ground; and 

defining a model of the object in the three-dimensional computer 
model in dependence upon the determined footprint. 

68. (Original) A method according to claim 67, wherein the step of 
processing the identified image data to determine the footprint of the object on the ground 
comprises: 

applying a transformation to the identified image data from the first 
camera which defines a mapping from the ground plane in the image data of the first 
camera to a surface in a modelling space; 

applying a transformation to the identified image data from the 
second camera which defines a mapping from the ground plane in the image data of the 
second camera to the surface in the modelling space; and 
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comparing the transformed image data on the surface in the 

modelling space. 

69. (Original) A method according to claim 68, wherein the surface in 
the modelling space is the ground plane in the three-dimensional computer model. 

70. (Original) A method according to claim 68, wherein the outline of 
the image on the ground is determined in dependence upon the aligned and non-aligned 
portions of the transformed image data on the surface in the modelling space. 

71. (Original) A method according to claim 67, wherein, the step of 
defining the model of the object comprises defining the model using a plurality of vertical 
planar surfaces. 

72. (Original) A method according to claim 71, wherein the vertical 
planar surfaces are defined such that their bases approximate the outline of the object on 
the ground. 

73. (Original) A method according to claim 71, wherein each planar 
surface is a rectangle. 
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74. (Original) A method according to claim 71, wherein each planar 
surface is defined with a height determined in dependence upon the image data identified 
from the first camera or the image data identified from the second camera. 

75. (Original) A method according to claim 74, wherein the height of 
each planar surface is defined in dependence upon a rectangle bounding some or all of the 
image data relating to the object identified from the first camera or the second camera. 

76. (Original) A method according to claim 74, wherein each planar 
surface is defined to have the same height. 

77. (Original) A method according to claim 71, further comprising the 
step of generating a top for the model of the object in dependence upon upper edges of the 
vertical planar surfaces. 

78. (Original) A method according to claim 67, further comprising the 
step of generating image data by rendering an image of the modelled object, in which 
texture data based on the identified image data from at least one camera is rendered onto 
the model. 

79. (Original) A method according to claim 78, wherein each planar 
surface is mapped onto the image data of the first camera or the second camera, and the 
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image data enclosed by each mapped surface is rendered onto the planar surface in the 
model. 



80. (Original) A method according to claim 78, further comprising the 
step of generating a signal conveying the image data. 

81 . (Original) A method according to claim 80, further comprising the 
step of recording the signal. 

82. (Original) A method according to claim 78, further comprising the 
step of displaying an image of the object using the generated image data. 

83. (Original) A method according to claim 78, further comprising the 
step of making a recording of the image data either directly or indirectly. 

84. (Original) A method of generating a model of an object in a 
three-dimensional computer model by processing images of the object from a plurality of 
cameras, in which image data from a first camera is processed to identify image data 
relating to the object, image data from a second camera is used to determine which parts of 
the identified image data from the first camera relate to parts of the object on or near the 
ground, and the object is represented in the computer model in dependence thereon. 
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85. (Original) Apparatus for processing image data defining a plurality 
of sequences of images, each from a respective camera, of an object moving in a scene to 
produce signals defining a representation of the object in a three-dimensional computer 
model, the apparatus comprising: 

means for processing image data from a first of the cameras to 
identify image data relating to the object in the scene; 

means for processing image data from a second of the cameras to 
identify image data relating to the object in the scene; 

means for processing the identified image data from the first camera 
and the identified image data from the second camera to determine a footprint of the object 
on the ground; and 

means for defining a model of the object in the three-dimensional 
computer model in dependence upon the determined footprint. 

86. (Original) Apparatus according to claim 85, wherein the means for 
processing the identified image data to determine the footprint of the object on the ground 
comprises: 

means for applying a transformation to the identified image data 
from the first camera which defines a mapping from the ground plane in the image data of 
the first camera to a surface in a modelling space; 

means for applying a transformation to the identified image data 
from the second camera which defines a mapping from the ground plane in the image data 
of the second camera to the surface in the modelling space; and 
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means for comparing the transformed image data on the surface in 

the modelling space. 

87. (Original) Apparatus according to claim 86, wherein the surface in 
the modelling space is the ground plane in the three-dimensional computer model. 

88. (Original) Apparatus according to claim 86, arranged to determine 
the outline of the image on the ground in dependence upon the aligned and non-aligned 
portions of the transformed image data on the surface in the modelling space. 

89. (Original) Apparatus according to claim 85, wherein, the means for 
defining the model of the object comprises means for defining the model using a plurality 
of vertical planar surfaces. 

90. (Original) Apparatus according to claim 89, arranged to perform 
processing such that the vertical planar surfaces are defined such that their bases 
approximate the outline of the object on the ground. 

91 . (Original) Apparatus according to claim 89, wherein each planar 
surface is a rectangle. 



92. (Original) Apparatus according to claim 89, arranged to perform 
processing such that each planar surface is defined with a height determined in dependence 
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upon the image data identified from the first camera or the image data identified from the 
second camera. 

93. (Original) Apparatus according to claim 92, arranged to perform 
processing such that the height of each planar surface is defined in dependence upon a 
rectangle bounding some or all of the image data relating to the object identified from the 
first camera or the second camera. 

94. (Original) Apparatus according to claim 92, wherein each planar 
surface is defined to have the same height. 

95. (Original) Apparatus according to claim 89, further comprising 
means for generating a top for the model of the object in dependence upon upper edges of 
the vertical planar surfaces. 

96. (Original) Apparatus according to claim 85, further comprising the 
step of generating image data by rendering an image of the modelled object, in which 
texture data based on the identified image data from at least one camera is rendered onto 
the model. 

97. (Original) Apparatus according to claim 96, arranged to perform 
processing such that each planar surface is mapped onto the image data of the first camera 
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or the second camera, and the image data enclosed by each mapped surface is rendered 
onto the planar surface in the model. 

98. (Original) Apparatus according to claim 96, further comprising 
means for displaying an image of the object using the generated image data. 

99. (Original) Apparatus for generating a model of an object in a 
three-dimensional computer model by processing images of the object from a plurality of 
cameras, the apparatus 

being operable to process image data from a first camera to identify image data relating to 
the object, to use image data from a second camera to determine which parts of the 
identified image data from the first camera relate to parts of the object on or near the 
ground, and to represent the object in the computer model in dependence thereon. 

100. (Currently Amended) A storage medium storing instructions for 
causing a programmable processing apparatus to become operable to perform a method 
according to any one of claims 67 and 84. 

101. (Currently Amended) A signal conveying physically-embodied 
computer program product including instructions in computer-readable form, including 
instructions for causing a programmable processing apparatus to become operable to 
perform a method according to any one of claims 67 and 84. 
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102. (Original) A method of processing image data defining a sequence 
of images of a plurality of objects moving in a scene to produce signals defining 
representations of the objects in a three-dimensional computer model, and to generate 
image data by rendering an image of the three-dimensional computer model in accordance 
with a user-selected viewing direction, the method comprising: 

processing the image data to identify image data relating to 
respective objects in the scene; 

defining a representation of each object in the three-dimensional 
computer model, in dependence upon the identified image data; and 

generating image data by rendering an image of the 
three-dimensional computer model in accordance with a user-selected viewing direction, 
wherein, when the selected viewing direction is within a predetermined range of viewing 
directions, texture data based on the identified image data is rendered onto the object 
representations, and, when the selected viewing direction is not within the predetermined 
range of viewing directions, a schematic of the positions of the objects in the scene is 
rendered. 

103. (Original) A method according to claim 102, wherein the 
representation of each object comprises a plurality of vertical planar surfaces alone. 

104. (Original) A method according to claim 102, wherein the 
representation of each object comprises a single vertical planar surface. 
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105. (Original) A method according to claim 102, wherein the 
predetermined range of viewing directions is a range relative to a fixed direction in the 
computer model. 

106. (Original) A method according to claim 105, wherein the 
predetermined range of viewing direction is a range relative to the vertical direction in the 
computer model. 

107. (Original) A method according to claim 102, wherein the 
predetermined range of viewing directions is a range relative to the representation of an 
object. 

108. (Original) A method according to claim 107, wherein the 
representation of an object comprises at least one vertical planar surface, and the 
predetermined range of viewing directions is a range relative to a planar surface. 

109. (Original) A method according to claim 102, wherein the schematic 
of the object positions is rendered from a predetermined viewing direction. 

110. (Original) A method according to claim 109, wherein the schematic 
is rendered from a vertical downward viewing direction. 
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111. (Original) A method according to claim 102, further comprising the 
steps of processing the image data to determine at least one colour for each object, and 
generating image 

data to indicate the determined colour on the schematic of the object positions. 

112. (Original) A method according to claim 102, further comprising the 
step of generating a signal conveying the image data. 

113. (Original) A method according to claim 112, further comprising the 
step of recording the signal. 

114. (Original) A method according to claim 102, further comprising the 
step of displaying an image of the objects using the generated image data. 

115. (Original) A method according to claim 102, further comprising the 
step of making a recording of the image data either directly or indirectly. 

116. (Original) A method of rendering an image in accordance with a 
user-selected viewing direction of a three-dimensional computer model comprising a 
representation and associated texture data for an object, the texture data being derived from 
image data recorded by at least one camera, the method comprising: 
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rendering the texture data onto the representation for the object in 
accordance with the user-selected viewing direction when the user-selected viewing 
direction is within a predetermined range of viewing directions; and 

rendering a schematic of the positions of the object when the 
user-selected viewing direction is not within the predetermined range of viewing 
directions. 

117. (Cancelled) 

118. (Original) Apparatus for processing image data defining a sequence 
of images of a plurality of objects moving in a scene to produce signals defining 
representations of the objects in a three-dimensional computer model, and to generate 
image data by rendering an image of the three-dimensional computer model in accordance 
with a user-selected viewing direction, the apparatus comprising: 

means for processing the image data to identify image data relating 
to respective objects in the scene; 

means for defining a representation of each object in the 
three-dimensional computer model, in dependence upon the identified image data; and 

means for generating image data by rendering an image of the 
three-dimensional computer model in accordance with a user-selected viewing direction, 
operable such that, when the selected viewing direction is within a predetermined range of 
viewing directions, texture data based on the identified image data is rendered onto the 
object representations, and, when the selected viewing direction is not within the 
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predetermined range of viewing directions, a schematic of the positions of the objects in 
the scene is rendered. 

1 19. (Original) Apparatus according to claim 118, operable to perform 
processing such that the representation of each object comprises a plurality of vertical 
planar surfaces alone. 

120. (Original) Apparatus according to claim 118, operable to perform 
processing such that the representation of each object comprises a single vertical planar 
surface. 

121. (Original) Apparatus according to claim 118, operable to perform 
processing such that the predetermined range of viewing directions is a range relative to a 
fixed direction in the computer model. 

122. (Original) Apparatus according to claim 121, operable to perform 
processing such that the predetermined range of viewing direction is a range relative to the 
vertical direction in the computer model. 

123. (Original) Apparatus according to claim 1 1 8, operable to perform 
processing such that the predetermined range of viewing directions is a range relative to the 
representation of an object. 
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124. (Original) Apparatus according to claim 123, operable to perform 
processing such that the representation of an object comprises at least one vertical planar 
surface, and the predetermined range of viewing directions is a range relative to a planar 
surface. 

125. (Original) Apparatus according to claim 118, operable to perform 
processing such that the schematic of the object positions is rendered from a predetermined 
viewing direction. 

126. (Original) A method according to claim 125, operable to perform 
processing such that the schematic is rendered from a vertical downward viewing direction. 

127. (Original) Apparatus according to claim 118, further comprising 
means for processing the image data to determine at least one colour for each object, and 
means for generating image data to indicate the determined colour on the schematic of the 
object positions. 

128. (Original) Apparatus according to claim 1 1 8, further comprising 
means for displaying an image of the objects using the generated image data. 

129. (Original) Apparatus for rendering an image in accordance with a 
user-selected viewing direction of a three-dimensional computer model comprising a 
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representation and associated texture data for an object, the texture data being derived from 
image data recorded by at least one camera, the apparatus comprising: 

means for rendering the texture data onto the representation for the 
object in accordance with the user-selected viewing direction when the user-selected 
viewing direction is within a predetermined range of viewing directions; and 

means for rendering a schematic of the positions of the object when 
the user-selected viewing direction is not within the predetermined range of viewing 
directions. 

130. (Cancelled) 

131. (Currently Amended) A storage medium storing instructions for 
causing a programmable processing apparatus to become operable to perform a method 
according to any one of claims 102 and 116. 

132. (Currently Amended) A signal conveying physically-embodied 
computer program product including instructions in computer-readable form, including 
instructions for causing a programmable processing apparatus to become operable to 
perform a method according to any one of claims 102 and 116. 

1 33. (Original) A method of processing image data defining a plurality of 
sequences of images, each from a respective camera, of an object moving in a scene to 
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produce signals defining a representation of the object in a three-dimensional computer 
model, the method comprising: 

processing image data from a first of the cameras to identify image 
data relating to the object in the scene; 

processing image data from a second of the cameras to identify 
image data relating to the object in the scene; 

processing the identified image data from the first camera and the 
identified image data from the second camera to identify planar surfaces on which points 
on the object lie by matching feature points in the identified image data from the first 
camera with feature points in the identified image data from the second camera, and 
identifying planar surfaces on which matched feature points lie; and 

defining a model of the object in the three-dimensional computer 
model in dependence upon the identified planar surfaces. 

134. (Original) A method according to claim 133, wherein corner points 
in the identified image data from the first camera are matched with corner points in the 
identified image data from the second camera. 

135. (Original) A method according to claim 133, wherein the planar 
surfaces are identified by identifying planes on which the matched feature points lie, and 
determining boundaries of the planes using matched feature points which lie on more than 
one plane. 
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136. (Original) A method according to claim 135, wherein each plane is 
identified by identifying a plane on which at least a predetermined number of feature points 
in the identified image data from the first camera lie, identifying the plane on which the 
matched feature points in the identified image data from the second camera lie, calculating 
a transformation between the plane in the image data from the first camera and the plane in 
the image data from the second camera, and testing the transformation using a plurality of 
other matched pairs of feature points. 

137. (Original) A method according to claim 136, wherein the 
predetermined number is four. 

138. (Original) A method according to claim 133, wherein the step of 
defining the model of the object comprises forming a model of planar surfaces in the 
three-dimensional computer model, each planar surface in the model corresponding to a 
planar surface identified in the image data from at least one of the cameras. 

139. (Original) A method according to claim 138, wherein the step of 
defining the model of the object comprises identifying a planar surface which touches the 
ground in the image data of a camera, defining a vertical planar surface in the 
three-dimensional computer model in dependence upon the identified planar surface which 
touches the ground, and defining a further planar surface in the three-dimensional computer 
model for each further planar surface in the image data of the camera such that the planar 
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surfaces in the three-dimensional computer model and the image data have the same aspect 
ratio. 

140. (Original) A method according to claim 139, wherein a planar 
surface which touches the ground in the image data of the given camera is identified by: 

applying a transformation to the base corner points of planar 
surfaces in the image data from the first camera which defines a mapping from the ground 
plane in the image data of the first camera to a surface in a modelling space; 

applying a transformation to the base corner points of planar 
surfaces in the image data from the second camera which defines a mapping from the 
ground plane in the image data of the second camera to the surface in the modelling space; 
and 

comparing the transformed corner points to determine which ones lie 
within a predetermined distance of each other. 

141 . (Original) A method according to claim 140, wherein the surface in 
the modelling space is the ground plane in the three-dimensional computer model. 

142. (Original) A method according to claim 141, wherein the defined 
vertical planar surface in the three-dimensional computer model is defined with a base 
defined by transformed corner points from the given camera which lie within the 
predetermined distance of the corresponding transformed corner points from the other 
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camera, and with an aspect ratio corresponding to the aspect ratio of the planar surface in 
the image data of the given camera to which the transformed corner points belong. 

143. (Original) A method according to claim 133, further comprising the 
step of generating image data by rendering an image of the modelled object, in which 
texture data based on the identified image data from at least one camera is rendered onto 
the model. 

144. (Original) A method according to claim 143, wherein image data 
enclosed by each planar surface is rendered on the corresponding planar surface of the 
object model. 

145. (Original) A method according to claim 143, further comprising the 
step of generating a signal conveying the image data. 

146. (Original) A method according to claim 145, further comprising the 
step of recording the signal. 

147. (Original) A method according to claim 143, further comprising the 
step of displaying an image of the object using the generated image data. 

148. (Original) A method according to claim 143, further comprising the 
step of making a recording of the image data either directly or indirectly. 
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149. (Original) A method of generating a model of an object in a 
three-dimensional computer model by processing images of the object from a plurality of 
cameras, in which image data from a first camera and a second camera is processed to 
match feature points in the image data from the first camera with feature points in image 
data from the second camera, the resulting matches are used to determine planar surfaces 
making up the object, and the object is represented in the computer model in dependence 
thereon. 

1 50. (Original) Apparatus for processing image data defining a plurality 
of sequences of images, each from a respective camera, of an object moving in a scene to 
produce signals defining a representation of the object in a three-dimensional computer 
model, the apparatus comprising: 

means for processing image data from a first of the cameras to 
identify image data relating to the object in the scene; 

means for processing image data from a second of the cameras to 
identify image data relating to the object in the scene; 

means for processing the identified image data from the first camera 
and the identified image data from the second camera to identify planar surfaces on which 
points on the object lie, comprising means for matching feature points in the identified 
image data from the first camera with feature points in the identified image data from the 
second camera, and means for identifying planar surfaces on which matched feature points 
lie; and 
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means for defining a model of the object in the three-dimensional 
computer model in dependence upon the identified planar surfaces. 

151. (Original) Apparatus according to claim 1 50, operable to perform 
processing such that corner points in the identified image data from the first camera are 
matched with corner points in the identified image data from the second camera. 

152. (Original) Apparatus according to claim 150, operable to perform 
processing such that the planar surfaces are identified by identifying planes on which the 
matched feature points lie, and determining boundaries of the planes using matched feature 
points which lie on more than one plane. 

153. (Original) Apparatus according to claim 152, operable to perform 
processing such that each plane is identified by identifying a plane on which at least a 
predetermined number of feature points in the identified image data from the first camera 
lie, identifying the plane on which the matched feature points in the identified image data 
from the second camera lie, calculating a transformation between the plane in the image 
data from the first camera and the plane in the image data from the second camera, and 
testing the transformation using a plurality of other matched pairs of feature points. 

154. (Original) Apparatus according to claim 153, wherein the 
predetermined number is four. 
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155. (Original) Apparatus according to claim 150, wherein the means for 
defining the model of the object comprises means for forming a model of planar surfaces in 
the three-dimensional computer model, each planar surface in the model corresponding to a 
planar surface identified in the image data from at least one of the cameras. 

156. (Original) Apparatus according to claim 155, wherein the means for 
defining the model of the object comprises means for identifying a planar surface which 
touches the ground in the image data of a camera, means for defining a vertical planar 
surface in the three-dimensional computer model in dependence upon the identified planar 
surface which touches the ground, and means for defining a further planar surface in the 
three-dimensional computer model for each further planar surface in the image data of the 
camera such that the planar surfaces in the three-dimensional computer model and the 
image data have the same aspect ratio. 

157. (Original) Apparatus according to claim 156, operable to perform 
processing such that a planar surface which touches the ground in the image data of the 
given camera is identified by: 

applying a transformation to the base corner points of planar 
surfaces in the image data from the first camera which defines a mapping from the ground 
plane in the image data of the first camera to a surface in a modelling space; 

applying a transformation to the base comer points of planar 
surfaces in the image data from the second camera which defines a mapping from the 
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ground plane in the image data of the second camera to the surface in the modelling space; 
and 

comparing the transformed corner points to determine which ones lie 
within a predetermined distance of each other. 

158. (Original) Apparatus according to claim 157, wherein the surface in 
the modelling space is the ground plane in the three-dimensional computer model. 

159. (Original) Apparatus according to claim 158, operable to perform 
processing such that the defined vertical planar surface in the three-dimensional computer 
model is defined with a base defined by transformed corner points from the given camera 
which lie within the predetermined distance of the corresponding transformed corner points 
from the other camera, and with an aspect ratio corresponding to the aspect ratio of the 
planar surface in the image data of the given camera to which the transformed corner points 
belong. 

160. (Original) Apparatus according to claim 150, further comprising 
means for generating image data by rendering an image of the modelled object, in which 
texture data based on the identified image data from at least one camera is rendered onto 
the model. 
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161. (Original) Apparatus according to claim 160, operable to perform 
processing such that image data enclosed by each planar surface is rendered on the 
corresponding planar surface of the object model. 

162. (Original) Apparatus according to claim 160, further comprising 
means for displaying an image of the object using the generated image data. 

163. (Original) Apparatus for generating a model of an object in a 
three-dimensional computer model by processing images of the object from a plurality of 
cameras, the apparatus being operable to process image data from a first camera and a 
second camera to match feature points in the image data from the first camera with feature 
points in the image data from the second camera, to use the resulting matches to determine 
planar surfaces making up the object, and to represent the object in the computer model in 
dependence thereon. 

164. (Currently Amended) A storage medium storing instructions for 
causing a programmable processing apparatus to become operable to perform a method 
according to any one of claims 133 and 149. 

165. (Currently Amended) A signal conveying physically-embodied 
computer program product including instructions in computer-readable form, including 
instructions for causing a programmable processing apparatus to become operable to 
perform a method according to any one of claims 133 and 149. 
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1 66. (Previously Presented) A method of processing image data defining 
a sequence of images of at least one object moving in a scene to produce signals defining a 
representation of each object in a three-dimensional computer model, and to generate 
image data by rendering an image of the three-dimensional computer model in accordance 
with a user-selected viewing direction, the method comprising: 

processing the image data to identify image data relating to 
respective objects in the scene; 

defining a representation of each object in the three-dimensional 
computer model in dependence upon the identified image data; 

generating image data by rendering an image of the three- 
dimensional computer model in accordance with a user-selected viewing direction in which 
texture data based on the identified image data is rendered onto the object representations; 
and 

generating reliability information indicating the reliability of the 
image data in dependence upon the angle between the user-selected viewing direction and 
the viewing direction from which the input image data was recorded. 

167. (Cancelled) 

168. (Previously Presented) A method according to claim 166, wherein 
the information indicating the reliability is generated in dependence upon a linear 
relationship between reliability and the angular difference between the user-selected 
viewing direction and the viewing direction from which the input image data was recorded. 
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169. (Previously Presented) A method according to claim 166, further 
comprising the step of generating information indicating how to change the viewing 
direction to improve the generated reliability. 

170. (Original) A method according to claim 166, wherein image data for 
a sequence of images recorded by a first camera and a sequence of images recorded by a 
second camera are processed such that: 

in the step of processing the image data, image data from the first 
camera relating to the respective objects in the scene is identified, and image data from the 
second camera relating to the respective objects in the scene is identified; 

in the step of defining a representation of each object, a first 
representation of each object is defined in dependence upon the identified image data from 
the first camera, and a second representation of each object is defined in dependence upon 
the identified image data from the second camera; and 

in the step of generating image data, texture data based on the 
identified image data from at least one camera is rendered onto the object representations. 

171. (Original) A method according to claim 166, wherein, in the step of 
defining a representation of each object, each object is represented as a planar surface. 

172. (Previously Presented) A method according to claim 166, wherein 
the reliability information is generated as pixel data within the generated image data. 
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173. (Previously Presented) A method according to claims 166, further 
comprising the step of generating a signal conveying the image data and the reliability 
information. 



174. (Original) A method according to claim 173, further comprising the 
step of recording the signal. 



175. (Previously Presented) A method according to claim 166, further 
comprising the step of displaying an image using the generated image data and displaying 
the reliability information. 

176. (Previously Presented) A method according to claim 166, further 
comprising the step of making a recording of the image data and the reliability information 
either directly or indirectly. 



177. (Previously Presented) A method of rendering an image in 
accordance with a user-selected viewing direction of a three-dimensional computer model 
comprising a representation and associated texture data for at least one object, the texture 
data being derived from image data recorded by at least one camera, the method 
comprising: 

generating image data by rendering an image of the three- 
dimensional computer model in accordance with a user-selected viewing direction, in 
which the texture data is rendered onto each representation; and 
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generating reliability information indicating the reliability of the 
image data in dependence upon the angle between the user-selected viewing direction and 
the viewing direction from which the input image data was recorded. 

178. (Previously Presented) An image processing method in which object 
data defining a three-dimensional computer model of at least one object in a scene is 
rendered in accordance with a user-selected viewing direction using image data recorded 
by a camera having a viewing direction to render each object, and an indicator of a quality 
of the generated image data is produced for output to the user in dependence upon the 
angle between the user-selected viewing direction and the viewing direction of the camera. 

1 79. (Previously Presented) Apparatus for processing image data defining 
a sequence of images of at least one object moving in a scene to produce signals defining a 
representation of each object in a three-dimensional computer model, and to generate 
image data by rendering an image of the three-dimensional computer model in accordance 
with a user-selected viewing direction, the apparatus comprising: 

means for processing the image data to identify image data relating 
to respective objects in the scene; 

means for defining a representation of each object in the three- 
dimensional computer model in dependence upon the identified image data; 

means for generating image data by rendering an image of the three- 
dimensional computer model in accordance with a user-selected viewing direction in which 
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texture data based on the identified image data is rendered onto the object representations; 
and 

means for generating reliability information indicating the reliability 
of the image data in dependence upon the angle between the user-selected viewing 
direction and the viewing direction from which the input image data was recorded. 

180. (Cancelled) 

181. (Previously Presented) Apparatus according to claim 1 79, operable 
to perform, processing such that the information indicating the reliability is generated in 
dependence upon a linear relationship between reliability and the angular difference 
between the user-selected viewing direction and the viewing direction from which the input 
image data was recorded. 

182. (Previously Presented) Apparatus according to claim 179, further 
comprising means for generating information indicating how to change the viewing 
direction to improve the generated reliability. 

183. (Original) Apparatus according to claim 179, operable to process 
image data for a sequence of images recorded by a first camera and a sequence of images 
recorded by a second camera, wherein: 
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the means for processing the image data is operable to identify 
image data from the first camera relating to the respective objects in the scene and to 
identify image data from the second camera relating to the respective objects in the scene; 

the means for defining a representation of each object is operable to 
define a first representation of each object in dependence upon the identified image data 
from the first camera, and to define a second representation of each object in dependence 
upon the identified image data from the second camera; and 

the means for generating image data is operable to render texture 
data based on the identified image data from at least one camera onto the object 
representations. 

184. (Original) Apparatus according to claim 179, wherein the means for 
defining a representation of each object is arranged to represent each object as a planar 
surface. 

1 85. (Previously Presented) Apparatus according to claim 1 79, operable 
to perform processing such that the reliability information is generated as pixel data within 
the generated image data. 

1 86. (Previously Presented) Apparatus according to claim 1 79, further 
comprising means for displaying an image using the generated image data and displaying 
the reliability information. 
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1 87. (Previously Presented) Apparatus for rendering an image in 
accordance with a user-selected viewing direction of a three-dimensional computer model 
comprising a representation and associated texture data for at least one object, the texture 
data being derived from image data recorded by at least one camera, the apparatus 
comprising: 

means for generating image data by rendering an image of the three- 
dimensional computer model in accordance with a user-selected viewing direction, in 
which the texture data is rendered onto each representation; and 

means for generating reliability information indicating the reliability 
of the image data in dependence upon the angle between the user-selected viewing 
direction and the viewing direction from which the input image data was recorded. 

188. (Previously Presented) An image processing apparatus operable to 
render object data defining a three-dimensional computer model of at least one object in a 
scene in accordance with a user-selected viewing direction using image data recorded by a 
camera having a viewing direction to render each object, and operable to produce an 
indicator of a quality of the generated image data for output to the user in dependence upon 
the angle between the user-selected viewing direction and the viewing direction of the 
camera. 

1 89. (Currently Amended) A storage medium storing instructions for 
causing a programmable processing apparatus to become operable to perform a method 
according to any one of claims 166, 177 and 178. 
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190. (Currently Amended) A signal conveying physically-embodied 
computer program product including instructions in computer-readable form, including 
instructions for causing a programmable processing apparatus to become operable to 
perform a method according to any one of claims 166, 177 and 178. 

191 . (Previously Presented) A method of processing image data defining 
a plurality of sequences of images, each from a respective camera, of an object moving in a 
scene to produce signals defining a representation of the object in a three-dimensional 
computer model, and to generate image data by rendering an image of the three- 
dimensional computer model in accordance with a user-selected viewing direction, the 
method comprising: 

receiving data defining a user-selected viewing direction; 
calculating the respective angle between the user-selected viewing direction and the 
respective viewing direction of each camera; 

identifying the cameras having a viewing direction within a 
predetermined angle of the user-selected viewing direction as identified cameras; 

comparing at least one camera characteristic affecting image data 
quality for each identified camera to determine differences therebetween; 

selecting one of the identified cameras as a selected camera in 
dependence upon the determined differences; 

processing input image data from the selected camera to define a 
representation of the object in the three-dimensional computer model; and 
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generating image data by rendering an image of the three- 
dimensional computer model in accordance with the user-selected viewing direction, in 
which texture data based on input image data from the selected camera is rendered onto the 
representation of the object. 

192. (Previously Presented) A method according to claim 191 or claim 
249, wherein, when comparing at least one camera characteristic affecting image data 
quality, at least one of the following are compared: the methods of transferring the image 
data from respective cameras; the resolution of respective cameras; the shutter speed of 
respective cameras; the stability of the image data from respective cameras; and whether 
the image data from respective cameras is colour or black and white. 

193. and 194. (Cancelled) 

195. (Previously Presented) A method according to claim 191 or claim 
249, wherein a plurality of camera characteristics affecting image quality are compared. 

196. (Previously Presented) A method according to claim 195, wherein 
the camera characteristics affecting quality are considered in a predetermined order and 
values for each respective camera characteristic are compared, with the selection of a 
camera being made once the tests identify a characteristic which differs by more than a 
predetermined amount for given cameras. 
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197. (Original) A method according to claim 191, further comprising the 
step of generating a signal conveying the image data. 

198. (Original) A method according to claim 197, further comprising the 
step of recording the signal. 

199. (Original) A method according to claim 191, further comprising the 
step of displaying an image of the objects using the generated image data. 

200. (Original) A method according to claim 191, further comprising the 
step of making a recording of the image data either directly or indirectly. 

201 . (Previously Presented) An image processing method in which image 
data from each of a respective sequence of images, each from a different camera, is 
processed to define a representation of at least one object in a three-dimensional computer 
model, and wherein a representation of each object is selected for rendering in dependence 
upon a user-selected viewing direction, the viewing direction of each camera and a 
plurality of camera parameters related to image data quality, wherein the plurality of 
camera parameters are tested in a predetermined order, with the selection of a 
representation for rendering being made once the tests identify a camera parameter which is 
sufficiently different for the cameras. 
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202. (Previously Presented) An image processing method in which a user- 
selected viewing direction in accordance with which an image of at least one object in a 
three-dimensional computer model is to be rendered is used to select, from among image 
data defining a plurality of images of the object each recorded by a respective camera, 
image data to be used to define the object in the three-dimensional computer model, the 
selection being carried out in dependence upon the user-selected viewing direction, 
together with the viewing direction of each camera and a plurality of camera parameters 
related to image data quality, wherein the plurality of camera parameters are tested in a 
predetermined order, with the selection of image data being made once the tests identify a 
camera parameter which is sufficiently different for the cameras. 

203. (Previously Presented) An image processing apparatus for 
processing image data defining a plurality of sequences of images, each from a respective 
camera, of an object moving in a scene to produce signals defining a representation of the 
object in a three-dimensional computer model, and to generate image data by rendering an 
image of the three-dimensional computer model in accordance with a user-selected viewing 
direction, the apparatus comprising: 

means for receiving data defining a user-selected viewing direction; 

means for calculating the respective angle between the user-selected 
viewing direction and the respective viewing direction of each camera; 

means for identifying the cameras having a viewing direction within 
a predetermined angle of the user-selected viewing direction as identified cameras; 
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means for comparing at least one camera characteristic affecting 
image data quality for each identified camera to determine differences therebetween; 

means for selecting one of the identified cameras as a selected 
camera in dependence upon the determined differences; 

means for processing input image data from the selected camera to 
define a representation of the object in the three-dimensional computer model; and 

means for generating image data by rendering an image of the three- 
dimensional computer model in accordance with the user-selected viewing direction, in 
which texture data based on input image data from the selected camera is rendered onto the 
representation of the object. 

204. (Previously Presented) Apparatus according to claim 203 or claim 
250, wherein the means for comparing at least one camera characteristic affecting image 
data quality is operable to compare at least one of: the methods of transferring the image 
data from respective cameras; the resolution of respective cameras; the shutter speed of 
respective cameras; the stability of the image data from respective cameras; and whether 
the image data from respective cameras is colour or black and white. 

205. and 206. (Cancelled) 

207. (Previously Presented) Apparatus according to claim 203 or claim 
250, operable to perform processing to compare a plurality of respective camera 
characteristics affecting image quality. 
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208. (Previously Presented) Apparatus according to claim 207, operable 
to perform processing such that the camera characteristics affecting quality are compared in 
a predetermined order, with the selection of a camera being made once the tests identify a 
characteristic which differs by more than a predetermined amount for given cameras. 

209. (Original) Apparatus according to claim 203, further comprising 
means for displaying an image of the objects using the generated image data. 

210. (Previously Presented) An image processing apparatus operable to 
process image data from each of a respective sequence of images, each from a different 
camera, to define a representation of at least one object in a three-dimensional computer 
model, and to select a representation of each object for rendering in dependence upon a 
user-selected viewing direction, the viewing direction of each camera and a plurality of 
camera parameters related to image data quality, the apparatus being operable to test the 
plurality of camera parameters in a predetermined order and to select a representation for 
rendering when the tests identify a camera parameter which is sufficiently different for the 
cameras. 

211. (Previously Presented) An image processing apparatus operable to 
use a user-selected viewing direction in accordance with which an image of at least one 
object in a three-dimensional computer model is to be rendered to select, from among 
image data defining a plurality of images of the object each recorded by a respective 
camera, image data to be used to define the object in the three-dimensional computer 
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model, the selection being carried out in dependence upon the user-selected viewing 
direction, together with the viewing direction of each camera and a plurality of camera 
parameters related to image data quality, wherein the apparatus is operable to test the 
plurality of camera parameters in a predetermined order and to select image data when the 
tests identify a camera parameter which is sufficiently different for the cameras. 

212. (Currently Amended) A storage medium storing instructions for 
causing a programmable processing apparatus to become operable to perform a method 
according to any one of claims 191, 201, 202 and 249. 

213. (Currently Amended) A signal conveying phvsicallv-embodied 
computer program product including instructions in computer-readable form, including 
instructions for causing a programmable processing apparatus become operable to perform 
a method according to any one of claims 191, 201, 202 and 249. 

2 1 4. (Original) A method of processing image data defining a plurality of 
sequences of images, each from a respective camera, of an object moving in a scene to 
produce signals defining a representation of the object in a three-dimensional computer 
model, and to generate image data for first and second images in a sequence of images of 
the object by rendering images of the three-dimensional computer model in accordance 
with first and second user-selected viewing directions, the method comprising: 

processing the image data to define at least one representation of the 
object in the three-dimensional computer model; 
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generating image data for use in a first image in the sequence by 
rendering texture data based on image data from at least a first of the cameras onto a 
representation of the object in accordance with a first user-selected viewing direction; 

generating image data for use in a second image in the sequence by 
rendering texture data based on image data from a second of the cameras onto a 
representation of the object in accordance with a second user-selected viewing direction; 

testing whether first and second images of the object displayed from 
the generated image data will be discontinuous by testing whether the image data for the 
object in the second image in the sequence differs by more than a predetermined amount 
from predetermined image data; and 

if the image data for the object in the second image differs by more 
than the predetermined amount, generating modified image data for the object in the 
second image. 

215. (Original) A method according to claim 214, wherein: 

in the step of processing the image data, image data from the first 
camera is processed to generate a first representation of the object in the three-dimensional 
computer model, and image data from the second camera is processed to generate a second 
representation of the object in the three-dimensional computer model; 

the image data for the object in the first image is generated by 
rendering the first representation; and 

the image data for the object in the second image is generated by 
rendering the second representation. 
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216. (Original) A method according to claim 215, wherein, in the step of 

testing: 

further image data for the object in the second image in the sequence 
is generated by rendering texture data based on image data from the first camera onto the 
first representation of the object in accordance with the second user-selected viewing 
direction; and 

the image data for the object in the second image generated using 
image data from the second camera is compared with the image data for the object in the 
second image generated using image data from the first camera. 

217. (Original) A method according to claim 216, wherein in the step of 
generating modified image data, the modified image data is generated in dependence upon 
the image data for the object in the second image generated using image data from the 
second camera and the image data for the object in the second image generated using 
image data from the first camera. 

218. (Original) A method according to claim 214, wherein: 

the step of generating image data for the first image comprises 
rendering the three-dimensional computer model in accordance with the first user-selected 
viewing direction; 

the step of generating image data for the second image comprises 
rendering the three-dimensional computer model in accordance with the second user- 
selected viewing direction; and 
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the step of testing comprises comparing the rendered image data for 
the second image with the predetermined image data. 

219. (Original) A method according to claim 214, further comprising the 
step of generating a signal conveying the modified image data. 

220. (Original) A method according to claim 219, further comprising the 
step of recording the signal. 

22 1 . (Original) A method according to claim 214, further comprising the 
step of displaying an image of the object using the modified image data. 

222. (Original) A method according to claim 214, further comprising the 
step of making a recording of the modified image data either directly or indirectly. 

223. (Original) A method of generating image data for first and second 
images in a sequence of images by rendering a three-dimensional computer model in 
accordance with respective first and second user-selected viewing directions, the three- 
dimensional computer model comprising a representation and associated texture data for at 
least one object and the texture data comprising texture data derived from image data 
recorded by a first camera and texture data derived from image data recorded by a second 
camera, the method comprising: 
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generating image data for use in a first image in the sequence by 
rendering texture data based on image data from at least a first camera onto the 
representation of each object in accordance with the first user-selected viewing direction; 

generating image data for use in the second image in the sequence by 
rendering texture data based on image data from the second camera onto the representation 
of each object in accordance with a second user-selected viewing direction; 

testing whether first and second images of the object displayed from 
the generated image data will be discontinuous by testing whether the image data for the 
object in the second image in the sequence differs by more than a predetermined amount 
from predetermined image data; and 

if the image data for the object in the second image differs by more 
than the predetermined amount, generating modified image data for the object in the 
second image. 

224. (Original) An image processing method in which a three- 
dimensional computer model including at least one representation of an object is processed 
a first time to generate image data for a first image in a sequence of images by rendering 
using image data recorded by a first camera as the basis for texture data for a 
representation, and a second time to generate image data for a successive image in the 
sequence by rendering using image data recorded by a second camera as the basis for 
texture data for a representation, and modified image data is generated for the object in the 
successive image if image data comparison tests indicate that the object in the images in 
the sequence will appear discontinuous. 
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225. (Original) A method of generating image data for successive images 
in a sequence by rendering a representation of an object in a three-dimensional computer 
model using image data from a plurality of cameras, in which a test on the image data is 
performed to determine whether the image of the object will appear discontinuous in the 
successive images, and the image data is processed to reduce the discontinuity. 

226. (Original) Image processing apparatus for processing image data 
defining a plurality of sequences of images, each from a respective camera, of an object 
moving in a scene to produce signals defining a representation of the object in a three- 
dimensional computer model, and to generate image data for first and second images in a 
sequence of images of the object by rendering images of the three-dimensional computer 
model in accordance with first and second user-selected viewing directions, the apparatus 
comprising: 

means for processing the image data to define at least one 
representation of the object in the three-dimensional computer model; 

means for generating image data for use in a first image in the 
sequence by rendering texture data based on image data from at least a first of the cameras 
onto a representation of the object in accordance with a first user-selected viewing 
direction; 

means for generating image data for use in a second image in the 
sequence by rendering texture data based on image data from a second of the cameras onto 
a representation of the object in accordance with a second user-selected viewing direction; 
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means for testing whether first and second images of the object 
displayed from the generated image data will be discontinuous by testing whether the 
image data for the object in the second image in the sequence differs by more than a 
predetermined amount from predetermined image data; and 

means for generating modified image data for the object in the 
second image if the image data for the object in the second image differs by more than the 
predetermined amount. 

227. (Original) Apparatus according to claim 226, wherein: 

the means for processing the image data is operable to process image 
data from the first camera to generate a first representation of the object in the three- 
dimensional computer model, and to process image data from the second camera to 
generate a second representation of the object in the three-dimensional computer model; 
and 

the apparatus is arranged to perform processing such that: 

- the image data for the object in the first image is generated 
by rendering the first representation; and 

- the image data for the object in the second image is 
generated by rendering the second representation. 

228. (Original) Apparatus according to claim 227, wherein the means for 
testing is operable to perform processing such that: 
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further image data for the object in the second image in the sequence 
is generated by rendering texture data based on image data from the first camera onto the 
first representation of the object in accordance with the second user-selected viewing 
direction; and 

the image data for the object in the second image generated using 
image data from the second camera is compared with the image data for the object in the 
second image generated using image data from the first camera. 

229. (Original) Apparatus according to claim 228, wherein the means for 
generating modified image data is arranged to perform processing such that the modified 
image data is generated in dependence upon the image data for the object in the second 
image generated using image data from the second camera and the image data for the 
object in the second image generated using image data from the first camera. 

230. (Original) Apparatus according to claim 226, wherein: 

the means for generating image data for the first image comprises 
means for rendering the three-dimensional computer model in accordance with the first 
user-selected viewing direction; 

the means for generating image data for the second image comprises 
means for rendering the three-dimensional computer model in accordance with the second 
user-selected viewing direction; and 

the means for testing comprises means for comparing the rendered 
image data for the second image with the predetermined image data. 
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23 1 . (Original) Apparatus according to claim 226, further comprising 
means for displaying an image of the object using the modified image data. 

232. (Original) Image processing apparatus for generating image data for 
first and second images in a sequence of images by rendering a three-dimensional 
computer model in accordance with respective first and second user-selected viewing 
directions, the three-dimensional computer model comprising a representation and 
associated texture data for at least one object and the texture data comprising texture data 
derived from image data recorded by a first camera and texture data derived from image 
data recorded by a second camera, the apparatus comprising: 

means for generating image data for use in a first image in the 
sequence by rendering texture data based on image data from at least a first camera onto 
the representation of each object in accordance with the first user-selected viewing 
direction; 

means for generating image data for use in the second image in the 
sequence by rendering texture data based on image data from the second camera onto the 
user-selected viewing direction; 

means for testing whether first and second images of the object 
displayed from the generated image data will be discontinuous by testing whether the 
image data for the object in the second image in the sequence differs by more than a 
predetermined amount from predetermined image data; and 
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means for generating modified image data for the object in the 
second image if the image data for the object in the second image differs by more than the 
predetermined amount. 

233. (Original) An image processing apparatus operable to process a 
three-dimensional computer model including at least one representation of an object a first 
time to generate image data for a first image in a sequence of images by rendering using 
image data recorded by a first camera as the basis for texture data for a representation, and 
a second time to generate image data for a successive image in the sequence by rendering 
using image data recorded by a second camera as the basis for texture data for a 
representation, and operable to generate modified image data for the object in the 
successive image if image data comparison tests indicate that the object in the images in 
the sequence will appear discontinuous. 

234. (Original) Apparatus for method of generating image data for 
successive images in a sequence by rendering a representation of an object in a three- 
dimensional computer model using image data from a plurality of cameras, the apparatus 
being operable to perform a test on the image data to determine whether the image of the 
object will appear discontinuous in the successive images, and to process the image data to 
reduce the discontinuity. 
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235. (Currently Amended) A storage medium storing instructions for 
causing a programmable processing apparatus to become operable to perform a method 
according to any one of claims 214, 223, 224 and 225. 

236. (Currently Amended) A signal conv e ying physically-embodied 
computer program product including instructions in computer-readable form, including 
instructions for causing a programmable processing apparatus to become operable to 
perform a method according to any one of claims 214, 223, 224 and 225. 

237. (Previously Presented) An image processing apparatus for 
processing image data defining a plurality of sequences of images, each from a respective 
camera, of a plurality of objects moving in a scene to produce signals defining 
representations of the objects in a three-dimensional computer model, the apparatus 
comprising: 

an image data identifier for processing image data from a first of the 
cameras to identify image data relating to objects in the scene, and for processing image 
data from a second of the cameras to identify image data relating to objects in the scene; 

an object representation definer for processing the identified image 
data from the first camera for each object to define an object representation in the three- 
dimensional computer model having a height dependent upon the image data for the object 
from the first camera, and for processing the identified image data from the second camera 
for each object to define an object representation in the three-dimensional computer model 
having a height dependent upon the image data for the object from the second camera; 
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a height comparer for comparing the height of the representation of 
each object generated in dependence upon image data from the first camera with the height 
of the representation of the corresponding object generated in dependence upon image data 
from the second camera; and 

an object representation generator for generating object 
representations in the three-dimensional computer model in dependence upon the height 
comparisons, 

wherein the apparatus is arranged to perform processing such that, 
when the heights of the corresponding representations are not within a predetermined 
amount of each other, the taller representation is split into a first portion having a height 
corresponding to the height of the smaller representation and a second portion comprising 
the remaining part of the taller representation, and wherein a further representation is 
defined in the three-dimensional model by re-positioning the second portion in the three- 
dimensional model. 

238. (Original) Apparatus for processing image data defining a plurality 
of sequences of images, each from a respective camera, of an object moving in a scene to 
produce signals defining a representation of the object in a three-dimensional computer 
model, the apparatus comprising: 

an image data identifier for processing image data from a first of the 
cameras to identify image data relating to the object in the scene, and for processing image 
data from a second of the cameras to identify image data relating to the object in the scene; 
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an image data transformer for applying a transformation to the 
identified image data from the first camera which defines a mapping from the ground plane 
in the space of the image data of the first camera to a surface in a modelling space, and for 
applying a transformation to the identified image data from the second camera which 
defines a mapping from the ground plane in the space of the image data of the second 
camera to the surface in the modelling space; 

an image data comparer for comparing the transformed image data 
from the first and second cameras on the surface in the modelling space; 

a shadow determinator for determining which part of the image data 
represents shadow in dependence upon the comparison results; and 

an object representation generator for generating a representation of 
at least the object in the three-dimensional model. 

239. (Previously Presented) Apparatus for generating a model of an 
object in a three-dimensional computer model, comprising: 

an image data transformer for applying a transformation to image 
data from a first camera relating to the object and its shadow which maps the image data to 
a surface to give first transformed image data, and for applying a transformation to image 
data from a second camera relating to the object and its shadow which maps the image data 
to the surface to give second transformed image data; 

a transformed image data comparer for comparing the first 
transformed image data and the second transformed image data to distinguish transformed 
image data relating to the object from transformed image data relating to its shadow; and 
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an object modeller for modelling the object in dependence upon the 
transformed image data relating to the object. 

240. (Original) Apparatus for processing image data defining a plurality 
of sequences of images, each from a respective camera, of an object moving in a scene to 
produce signals defining a representation of the object in a three-dimensional computer 
model, the apparatus comprising: 

an image data identifier for processing image data from a first of the 
cameras to identify image data relating to the object in the scene, and for processing image 
data from a second of the cameras to identify image data relating to the object in the scene; 

a footprint determinator for processing the identified image data 
from the first camera and the identified image data from the second camera to determine a 
footprint of the object on the ground; and 

an object modeller for defining a model of the object in the three- 
dimensional computer model in dependence upon the determined footprint. 

241 . (Original) Apparatus for processing image data defining a sequence 
of images of a plurality of objects moving in a scene to produce signals defining 
representations of the objects in a three-dimensional computer model, and to generate 
image data by rendering an image of the three-dimensional computer model in accordance 
with a user-selected viewing direction, the apparatus comprising: 
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an image data identifier for processing the image data to identify 
image data relating to respective objects in the scene; 

an object modeller for defining a representation of each object in the 
three-dimensional computer model, in dependence upon the identified image data; and 

a renderer for generating image data by rendering an image of the 
three-dimensional computer model in accordance with a user-selected viewing direction, 
operable such that, when the selected viewing direction is within a predetermined range of 
viewing directions, texture data based on the identified image data is rendered onto the 
object representations, and, when the selected viewing direction is not within the 
predetermined range of viewing directions, a schematic of the positions of the objects in 
the scene is rendered. 

242. (Original) Apparatus for rendering an image in accordance with a 
user-selected viewing direction of a three-dimensional computer model comprising a 
representation and associated texture data for an object, the texture data being derived from 
image data recorded by at least one camera, the apparatus comprising: 

a first renderer for rendering the texture data onto the representation 
for the object in accordance with the user- selected viewing direction when the user- 
selected viewing direction is within a predetermined range of viewing directions; and 

a second renderer for rendering a schematic of the positions of the 
object when the user-selected viewing direction is not within the predetermined range of 
viewing directions. 
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243. (Original) Apparatus for processing image data defining a plurality 
of sequences of images, each from a respective camera, of an object moving in a scene to 
produce signals defining a representation of the object in a three-dimensional computer 
model, the apparatus comprising: 

an image data identifier for processing image data from a first of the 
cameras to identify image data relating to the object in the scene, and for processing image 
data from a second of the cameras to identify image data relating to the object in the scene; 

a surface identifier for processing the identified image data from the 
first camera and the identified image data from the second camera to identify planar 
surfaces on which points on the object lie, comprising a feature matcher for matching 
feature points in the identified image data from the first camera with feature points in the 
identified image data from the second camera, and a planar surface identifier for 
identifying planar surfaces on which matched feature points lie; and 

an object modeller for defining a model of the object in the three- 
dimensional computer model in dependence upon the identified planar surfaces. 

244. (Previously Presented) Apparatus for processing image data defining 
a sequence of images of at least one object moving in a scene to produce signals defining a 
representation of each object in a three-dimensional computer model, and to generate 
image data by rendering an image of the three-dimensional computer model in accordance 
with a user-selected viewing direction, the apparatus comprising: 

an image data identifier for processing the image data to identify 
image data relating to respective objects in the scene; 
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an object modeller for defining a representation of each object in the 
three-dimensional computer model in dependence upon the identified image data; 

a renderer for generating image data by rendering an image of the 
three-dimensional computer model in accordance with a user-selected viewing direction in 
which texture data based on the identified image data is rendered onto the object 
representations; and 

a reliability information generator for generating reliability 
information indicating the reliability of the image data in dependence upon the angle 
between the user-selected viewing direction and the viewing direction from which the input 
image data was recorded. 

245. (Previously Presented) Apparatus for rendering an image in 
accordance with a user-selected viewing direction of a three-dimensional computer model 
comprising a representation and associated texture data for at least one object, the texture 
data being derived from image data recorded by at least one camera, the apparatus 
comprising: 

a Tenderer for generating image data by rendering an image of the 
three-dimensional computer model in accordance with a user-selected viewing direction, in 
which the texture data is rendered onto each representation; and 

a reliability information generator for generating reliability 
information indicating the reliability of the image data in dependence upon the angle 
between the user-selected viewing direction and the viewing direction from which the input 
image data was recorded. 

-72- 



246. (Previously Presented) An image processing apparatus for 
processing image data defining a plurality of sequences of images, each from a respective 
camera, of an object moving in a scene to produce signals defining a representation of the 
object in a three-dimensional computer model, and to generate image data by rendering an 
image of the three-dimensional computer model in accordance with a user-selected viewing 
direction, the apparatus comprising: 

a data receiver for receiving data defining a user-selected viewing 

direction; 

an angle calculator operable to calculate the respective angle 
between the user-selected viewing direction and the respective viewing direction of each 
camera; 

a camera identifier operable to identify the cameras having a viewing 
direction within a predetermined angle of the user-selected viewing direction as identified 
cameras; 

a camera characteristic comparer operable to compare at least one 
camera characteristic affecting image data quality for each identified camera to determine 
differences therebetween; 

a camera selector operable to select one of the identified cameras as 
a selected camera in dependence upon the determined differences; 

an object representation generator for processing input image data 
from the selected camera to define a representation of the object in the three-dimensional 
computer model; and 
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a renderer for generating image data by rendering an image of the 
three-dimensional computer model in accordance with the user-selected viewing direction, 
in which texture data based on input image data from the selected camera is rendered onto 
the representation of each object. 

247. (Original) Image processing apparatus for processing image data 
defining a plurality of sequences of images, each from a respective camera, of an object 
moving in a scene to produce signals defining a representation of the object in a three- 
dimensional computer model, and to generate image data for first and second images in a 
sequence of images of the object by rendering images of the three-dimensional computer 
model in accordance with first and second user-selected viewing directions, the apparatus 
comprising: 

an object representation generator for processing the image data to 
define at least one representation of the object in the three-dimensional computer model; 

a renderer for generating image data for use in a first image in the 
sequence by rendering texture data based on image data from at least a first of the cameras 
onto a representation of the object in accordance with a first user-selected viewing 
direction, and for generating image data for use in a second image in the sequence by 
rendering texture data based on image data from a second of the cameras onto a 
representation of the object in accordance with a second user-selected viewing direction; 

an image data tester for testing whether first and second images of 
the object displayed from the generated image data will be discontinuous by testing 
whether the image data for the object in the second image in the sequence differs by more 
than a predetermined amount from predetermined image data; and 
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an image data modifier for generating modified image data for the 
object in the second image if the image data for the object in the second image differs by 
more than the predetermined amount. 

248. (Original) Image processing apparatus for generating image data for 
first and second images in a sequence of images by rendering a three-dimensional 
computer model in accordance with respective first and second user-selected viewing 
directions, the three-dimensional computer model comprising a representation and 
associated texture data for at least one object and the texture data comprising texture data 
derived from image data recorded by a first camera and texture data derived from image 
data recorded by a second camera, the apparatus comprising: 

a renderer for generating image data for use in a first image in the 
sequence by rendering texture data based on image data from at least a first camera onto 
the representation of each object in accordance with the first user-selected viewing 
direction, and for generating image data for use in the second image in the sequence by 
rendering texture data based on image data from the second camera onto the representation 
of each object in accordance with a second user-selected viewing direction; 

an image data tester for testing whether first and second images of 
the object displayed from the generated image data will be discontinuous by testing 
whether the image data for the object in the second image in the sequence differs by more 
than a predetermined amount from predetermined image data; and 
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an image data modifier for generating modified image data for the 
object in the second image if the image data for the object in the second image differs by 
more than the predetermined amount. 

249. (Previously Presented) A method of processing image data defining 
a plurality of sequences of images, each from a respective camera, of an object moving in a 
scene to produce signals defining a representation of the object in a three-dimensional 
computer model, and to generate image data by rendering an image of the three- 
dimensional computer model in accordance with a user-selected viewing direction, the 
method comprising: 

processing input image data from each respective camera to define a 
respective representation of the object in the three-dimensional computer model; 

receiving data defining a user-selected viewing direction; 

calculating the respective angle between the user-selected viewing 
direction and the respective viewing direction of each camera; 

identifying the cameras having a viewing direction within a 
predetermined angle of the user-selected viewing direction as identified cameras; 

comparing at least one camera characteristic affecting the image data 
quality for each identified camera to determine differences therebetween; 

selecting one of the identified cameras as a selected camera in 
dependence upon the determined differences; 

selecting the representation of the object generated from the selected 
camera as a selected representation; and 
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generating image data by rendering an image of the three- 
dimensional computer model in accordance with the user-selected viewing direction, in 
which texture data based on input image data from the selected camera is rendered onto the 
selected representation of the object. 

250. (Previously Presented) An image processing apparatus for 
processing image data defining a plurality of sequences of images, each from a respective 
camera, of an object moving in a scene to produce signals defining a representation of the 
object in a three-dimensional computer model, and to generate image data by rendering an 
image of the three-dimensional computer model in accordance with a user-selected viewing 
direction, the apparatus comprising: 

means for processing input image data from each respective camera 
to define a respective representation of the object in the three-dimensional computer 
model; 

means for receiving data defining a user-selected viewing direction; 

means for calculating the respective angle between the user-selected 
viewing direction and the respective viewing direction of each camera; 

means for identifying the cameras having a viewing direction within 
a predetermined angle of the user-selected viewing direction as identified cameras; 

means for comparing at least one camera characteristic affecting the 
image data quality for each identified camera to determine differences therebetween; 

means for selecting one of the identified cameras as a selected 
camera in dependence upon the determined differences; 
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means for selecting the representation of the object generated from 
the selected camera as a selected representation; and 

means for generating image data by rendering an image of the three- 
dimensional computer model in accordance with the user-selected viewing direction, in 
which texture data based on input image data from the selected camera is rendered onto the 
selected representation of the object. 

251. (Previously Presented) An image processing apparatus for 
processing image data defining a plurality of sequences of images, each from a respective 
camera, of an object moving in a scene to produce signals defining a representation of the 
object in a three-dimensional computer model, and to generate image data by rendering an 
image of the three-dimensional computer model in accordance with a user-selected viewing 
direction, the apparatus comprising: 

an object representation generator operable to process input image 
data from each respective camera to define a respective representation of the object in the 
three-dimensional computer model; 

a data receiver for receiving data defining a user-selected viewing 

direction; 

an angle calculator operable to calculate the respective angle 
between the user-selected viewing direction and the respective viewing direction of each 
camera; 
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a camera identifier operable to identify the cameras having a viewing 
direction within a predetermined angle of the user-selected viewing direction as identified 
cameras; 

a camera characteristic comparer operable to compare at least one 
camera characteristic affecting the image data quality for each identified camera to 
determine differences therebetween; 

a camera selector operable to select one of the identified cameras as 
a selected camera in dependence upon the determined differences; 

an object representation selector operable to select the representation 
of the object generated from the selected camera as a selected representation; and 

a renderer operable to generate image data by rendering an image of 
the three-dimensional computer model in accordance with the user-selected viewing 
direction, in which texture data based on input image data from the selected camera is 
rendered onto the selected representation of the object. 

252. (Previously Presented) A method of processing image data defining 
a plurality of sequences of images, each from a respective camera, of a plurality of objects 
moving in a scene to produce signals defining representations of the objects in a three- 
dimensional computer model, the method comprising the steps of: 

processing image data from a first of the cameras to identify image 
data relating to objects in the scene; 

processing image data from a second of the cameras to identify 
image data relating to objects in the scene; 
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processing the identified image data from the first camera for each 
object to define an object representation comprising a planar surface with its base on a 
predetermined surface in a modelling space having a position, width and height dependent 
upon a polygon bounding the image data for the object from the first camera, wherein the 
width of each planar surface representing an object is determined by the width of the 
associated bounding box in the image data, and the height of each planar surface is 
calculated using the aspect ratio of the bounding box in the image data; 

processing the identified image data from the second camera for 
each object to define an object representation comprising a planar surface with its base on a 
predetermined surface in the modelling space having a position, width and height 
dependent upon a polygon bounding the image data for the object from the second camera, 
wherein the width of each planar surface representing an object is determined by the width 
of the associated bounding box in the image data, and the height of each planar surface is 
calculated using the aspect ratio of the bounding box in the image data; 

comparing the height of the representation of each object generated 
in dependence upon image data from the first camera with the height of the representation 
of the corresponding object generated in dependence upon image data from the second 
camera; and 

generating object representations in the three-dimensional computer 
model in dependence upon the height comparisons. 

253. (Currently Amended) An image processing apparatus for processing 
image data defining a plurality of sequences of images, each from a respective camera, of a 
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plurality of objects moving in a scene to produce signals defining representations of the 
objects in a three-dimensional computer model, the apparatus comprising: 

means for processing image data from a first of the cameras to 
identify image data relating to objects in the scene; 

means for processing image data from a second of the cameras to 
identify image data relating to objects in the scene; 

means for processing the identified image data from the first camera 
for each object to define an object representation comprising a planar surface with its base 
on a predetermined surface in a modelling space having a position, width and height 
dependent upon a polygon bounding the image data for the object from the first camera, 
wherein the width of each planar surface representing an object is determined by the width 
of the associated bounding box in the image data, and the height of each planar surface is 
calculated using the aspect ratio of the bounding box in the image data; 

means for processing the identified image data from the second 
camera for each object to define an object representation comprising planar surface a 
planar surface with its base on a predetermined surface in the modelling space having a 
position, width and height dependent upon a polygon bounding the image data for the 
object from the second camera, wherein the width of each planar surface representing an 
object is determined by the width of the associated bounding box in the image data, and the 
height of each planar surface is calculated using the aspect ratio of the bounding box in the 
image data; 

means for comparing the height of the representation of each object 
generated in dependence upon image data from the first camera with the height of the 
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representation of the corresponding object generated in dependence upon image data from 
the second camera; and 

means for generating object representations in the three-dimensional 
computer model in dependence upon the height comparisons. 

254. (Previously Presented) An image processing apparatus for 
processing image data defining a plurality of sequences of images, each from a respective 
camera, of a plurality of objects moving in a scene to produce signals defining 
representations of the objects in a three-dimensional computer model, the apparatus 
comprising: 

an image data identifier for processing image data from a first of the 
cameras to identify image data relating to objects in the scene, and for processing image 
data from a second of the cameras to identify image data relating to objects in the scene; 

an object representation definer for processing the identified image 
data from the first camera for each object to define an object representation comprising a 
planar surface with its base on a predetermined surface in a modelling space having a 
position, width and height dependent upon a polygon bounding the image data for the 
object from the first camera, and for processing the identified image data from the second 
camera for each object to define an object representation comprising a planar surface with 
its base on a predetermined surface in the modelling space having a position, width and 
height dependent upon a polygon bounding the image data for the object from the second 
camera, 
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wherein the object representation definer is arranged to perform 
processing such that the width of each planar surface representing an object is determined 
by the width of the associated bounding box in the image data, and the height of each 
planar surface is calculated using the aspect ratio of the bounding box in the image data; 

a height comparer for comparing the height of the representation of 
each object generated in dependence upon image data from the first camera with the height 
of the representation of the corresponding object generated in dependence upon image data 
from the second camera; and 

an object representation generator for generating object 
representations in the three-dimensional computer model in dependence upon the height 
comparisons. 
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